A solid image pickup device mounted in an imaging apparatus such as a digital camera has reached a high level of resolution, and for example, even if printing in a large size is performed, a high picture quality can be obtained. On the other hand, the resolution of a display device such as a liquid crystal display device has not been improved as compared with the solid image pickup device. Besides, from the beginning, since the display mounted on the imaging apparatus is relatively small and is provided for use in confirmation of a shot image, very high resolution is not required.
Also in cellular phones with camera function, there appears one equipped with a high resolution solid image pickup device. However, in the case where an image photographed by that is transmitted by wireless, an image with low resolution is transmitted from the restriction of transmission capacity. Besides, in a digital camera or the like, an image with high resolution is recorded in a still image shooting mode, while an image with low resolution is recorded in a motion picture shooting mode from the restriction of recording capacity.
Further, with respect to an image format, there are standards such as VGA (Video Graphics Array) and CIF (Common Intermediate Format). Specifically, the VGA size is 640 dots horizontally by 480 dots vertically, and the CIF size is 352 dots horizontally by 288 dots vertically. Besides, the respective standards include QVGA size, QCIF size and the like. For example, the CIF is often adopted in the motion picture photographing.
As described above, although the solid image pickup device has reached a high level of resolution, an image signal outputted from the device is not always used as it is, and is often converted into another format and is used. For example, in the case where an image taken in the VGA size is converted into the CIF size, it is necessary to perform scaling of ⅗ with reference to the vertical size.
Conventionally, in the scaling performed in a digital camera or the like, luminance data Y and color-difference data U and V generated by a digital signal processing circuit on the basis of an image signal taken by a solid image pickup device are once stored in a frame memory or the like. The scaling is performed by carrying out an interpolation operation or the like on the image data stored in this memory. For example, image data of two adjacent lines before conversion are linearly interpolated, so that image data of a line positioned therebetween in an image after conversion is generated.
However, when one screen is once stored in the memory and the scaling processing is performed, a delay time due to the image storage into the memory occurs. Thus, there is a problem that a time lag occurs and operationality is lowered in a situation in which a real-time property is required, such as a format conversion at the time of display on a preview screen or a zooming processing during motion picture photographing.
Then, the real-time property of the image format conversion processing is improved by a structure shown in FIG. 1. FIG. 1 is a block diagram of a conventional imaging apparatus in which a photographed VGA image is converted into a CIF image and is outputted. This imaging apparatus includes a CCD image sensor 2 and an image signal processing circuit for processing an image signal Y0(t) outputted by the CCD image sensor 2. The image signal processing circuit includes an analog signal processing circuit 4, an ADC (analog-to-digital converter) 6, a digital signal processing circuit 8, and a format conversion circuit 10.
The CCD image sensor 2 includes color filters of Bayer arrangement in which color components are changed at a period of two pixels in each of the line direction and the column direction, and outputs the VGA image signal Y0(t).
Image data D(n) at each pixel of the VGA image is inputted to the digital signal processing circuit 8. The digital signal processing circuit 8 includes a VLPF 20 and a HLPF 22 in order to remove noise which causes moire. The VLPF 20 is a filter for trapping a frequency component of ½ of a vertical sampling frequency, and the HLPF 22 is a filter for trapping a frequency component of ½ of a horizontal sampling frequency. Line memories 24 (24-1 to 24-5) for holding five lines of image data used in the VLPF 20 are connected to the input of the VLPF 20. As described later, the VLPF 20 outputs three kinds of image data IY, IC1 and IC2. The image data IY, IC1 and IC2 are inputted to a signal processing part 26 through the HLPF 22. The signal processing part 26 performs signal processing such as color separation, gamma correction and contour correction. The RGB data outputted from the signal processing part 26 are inputted to a YUV generation part 28. The YUV generation part 28 generates luminance data Y and color-difference data U and V on the basis of the RGB data.
The data Y, U and V outputted from the digital signal processing circuit 8 have respectively VGA resolution, and the format conversion circuit 10 converts these into luminance data Y′, and color-difference data U′ and V′ of CIF resolution. In the case where a line of a CIF image is defined between two adjacent lines of a VGA image, the format conversion circuit 10 linearly interpolates image data of two lines of the VGA image, and generates image data of the CIF image corresponding to the line. The format conversion circuit 10 includes line memories 30 (30-1, 30-2) for holding the two adjacent lines of the luminance data Y of the VGA image. An interpolation processing part 36-1 interpolates the two lines, and generates the luminance signal Y′ of the CIF image. Besides, the format conversion circuit 10 includes two line memories 34 (34-1, 34-2) and an interpolation processing part 36-2 in order to perform a similar interpolation processing on the two kinds of color-difference data U and V. The line memories 34 and the interpolation processing part 36-2 are shared between the processings of the two kinds of color-difference data.
FIG. 2 is a circuit diagram showing a circuit structure of the VLPF 20. Image data D (hereinafter, image data of a kth line is referred to as D(k)) of five lines (jth line to (j+4)th line), which are continuous in the VGA image and are respectively held in the five line memories 24, are inputted in parallel to the VLPF 20. The VLPF 20 generates three kinds of image data IY, IC1 and IC2 corresponding to a center line position of the five lines. The image data held in the line memories 24 are exchanged in synchronization with the output of the CCD image sensor 2, and in response to that, the image data IY, IC1 and IC2 are obtained in sequence for each line. The VLPF 20 weights the respective data of the jth to (j+4)th lines. Here, a weighting factor for each of them is determined so that the filter characteristic of trapping the frequency component of ½ of the vertical sampling frequency is obtained as described above, and for example, the weighting factors can be respectively made 1, 6, 10, 6 and 1 for the jth to (j+4)th lines. In order to perform the weighting, the image data D of the (j+1)th to (j+3)th lines are respectively inputted to multipliers 40-1 to 40-3 whose multiplication factors are set to 6, 10 and 6. An adder 42-1 adds the weighted image data of five lines and generates the image data IY. An adder 42-2 adds the weighted image data of the (j+1)th line and the (j+3)th line and generates the image data IC1. An adder 42-3 adds the weighted image data of the jth, (j+2)th and (j+4)th lines and generates the image data IC2. That is, with respect to the (j+2)th line of the VGA image, three kinds of image data are generated:IY(j+2)=D(j)+6D(j+1)+10D(j+2)+6D(j+3)+D(j+4)IC1(j+2)=6D(j+1)+6D(j+3)IC2(j+2)=D(j)+10D(j+2)+D(j+4)
As described above, the format conversion to change the vertical resolution, such as the conversion from VGA to CIF, is conventionally performed by the interpolation processing in the vertical direction. In order to perform this interpolation processing in real time, the line memories for holding the two adjacent lines of image data generated in the formation before conversion are provided. Since this interpolation processing is performed on each line of the original image after plural kinds of image data, such as the luminance data Y and the color-difference data U and V, are obtained, in order to process them simultaneously and in parallel, plural pairs of line memories are required like the line memories 30 and 34 as set forth in the above example.
Here, in the case where an imaging apparatus is mounted in a small equipment such as a cellular phone, further miniaturization of an image processing apparatus is required. Thus, there has been a problem to omit the line memories 30 and 34 used for the format conversion, and to reduce constituent parts, to reduce the chip area of an integrated circuit, and to reduce the cost.
Incidentally, JP-A-9-98437 discloses an imaging apparatus using a solid image pickup device including a mosaic filter. The publication relates to a technique to generate image data in which moire is suppressed, and discloses a related technique to generate image data of each line of a VGA image on the basis of image data of plural lines outputted from the solid image pickup device.